Signal Estimation in Gaussian Noise: A Statistical Physics Perspective

We consider the problem of signal estimation from a statistical mechanical perspective, using a relationship between the minimum mean square error (MMSE), of estimating a signal, and the mutual information between this signal and its noisy version. We derive several statistical-mechanical relationships between a few important quantities in this problem area, such as the MMSE, the mutual information, the Fisher information, the free energy, and a generalized notion of temperature. We also draw analogies and differences between certain relations pertaining to the estimation problem and the parallel relations in statistical physics. Finally, we provide several examples, demonstrating how analysis tools in statistical physics prove useful in the MMSE analysis. In most of the examples, the corresponding statisticalmechanical systems turn out to exhibit phase transitions, which are reflected as irregularities in behavior of the MMSE.